Serial Concatenation of RS Codes with Kite 
Codes: Performance Analysis, Iterative 
Decoding and Design 

Xiao Ma, Kai Zhang, Baoming Bai and Xiaoyi Zhang 

Abstract 

In this paper, we propose a new ensemble of rateless forward error correction (FEC) codes. The 
proposed codes are serially concatenated codes with Reed-Solomon (RS) codes as outer codes and Kite 
codes as inner codes. The inner Kite codes are a special class of prefix rateless low-density parity- 
check (PRLDPC) codes, which can generate potentially infinite (or as many as required) random-like 
parity-check bits. The employment of RS codes as outer codes not only lowers down error-floors but also 
ensures (with high probability) the correctness of successfully decoded codewords. In addition to the 
conventional two-stage decoding, iterative decoding between the inner code and the outer code are also 
implemented to improve the performance further. The performance of the Kite codes under maximum 
likelihood (ML) decoding is analyzed by applying a refined Divsalar bound to the ensemble weight 
enumerating functions (WEF). We propose a simulation-based optimization method as well as density 
evolution (DE) using Gaussian approximations (GA) to design the Kite codes. Numerical results along 
with semi-analytic bounds show that the proposed codes can approach Shannon limits with extremely 
low error-floors. It is also shown by simulation that the proposed codes performs well within a wide 
range of signal-to-noise-ratios (SNRs). 
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I. Introduction 

Serially concatenated codes were first introduced by Forney [QQ|. A traditional concatenated 
code typically consists of an outer Reed-Solomon (RS) code, an inner convolutional code and a 
symbol interleaver in between. A typical example is the concatenation of an outer RS[255, 223, 33] 
code and an inner convolutional code with rate 0.5 and constraint length 7, which has been 
adopted as the CCSDS Telemetry Standard [2J. The basic idea behind the concatenated codes is 
to build long codes from shorter codes with manageable decoding complexity. The task of the 
outer RS decoder is to improve the performance further by correcting "remaining" errors after 
the inner decoding. Since the invention of turbo codes [3] and the rediscovery of low-density 
parity-check (LDPC) codes [4], it has been suggested to use serially concatenated codes with 
outer RS codes and inner turbo/LDPC codes for data transmissions BSKSJ. 

Binary linear rateless coding is an encoding method that can generate potentially infinite 
parity-check bits for any given fixed-length binary sequence. Fountain codes constitute a class 
of rateless codes, which were first mentioned in EJ. For a conventional channel code, coding 
rate is well defined; while for a Fountain code, coding rate is meaningless since the number of 
coded bits is potentially infinite and may be varying for different applications. The first practical 
realizations of Fountain codes were LT-codes invented by Luby |[8). LT-codes are linear rateless 
codes that transform k information bits into infinite coded bits. Each coded bit is generated 
independently and randomly as a binary sum of several randomly selected information bits, 
where the randomness is governed by the so-called robust soliton distribution. LT-codes have 
encoding complexity of order log A; per information bit on average. The complexity is further 
reduced by Shokrollahi using Raptor codes [9 |. The basic idea behind Raptor codes is to precode 
the information bits prior to the application of an appropriate LT-code. For relations among 
random linear Fountain codes, LT-codes and Raptor codes and their applications over erasure 
channels, we refer the reader to [TTOl . 

Fountain codes were originally proposed for binary erasure channels (BECs). In addition to 
their success in erasure channels, Fountain codes have also been applied to other noisy binary 
input memory less symmetric channels (BIMSCs) [fTT1[[T2l[fT3l . Comparing other noisy BIMSCs 
with BECs, we have the following observations. 

1) Over erasure channels, Fountain codes can be used in an asynchronous way as follows. 
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The transmission bits are first grouped into packets. Then LT-codes and/or Raptor codes 
are adapted to work in a packet-oriented way which is similar to the bit-oriented way. The 
only difference lies in that the encoders can insert a packet-head to each coded packet, 
specifying the involved information packets. While the receiver receives a coded packet, it 
also knows the connections of this coded packets to information packets. In this scenario, the 
receiver does not necessarily know the code structure before the transmission. The overhead 
due to the insertion of packet-head can be made negligible whenever the size of the packets 
is large. However, over other noisy channels, it should be more convenient and economic 
to consider only synchronous Fountain codes. This is equivalent to require that the receiver 
knows the structure of the code before transmission or shares common randomness with the 
transmitter. Otherwise, one must find ways to guarantee with probability 1 the correctness 
of the packet-head. 

2) Both LT-codes and Raptor codes have been proved to be universally good for erasure 
channels in the sense that, no matter what the channel erasure probability is, the iterative 
belief propagation (BP) algorithm can recover the transmitted k bits (packets) with high 
probability from any received k(l + e) bits (packets). However, it has been proved [12] that 
no universal Raptor codes exist for BIMSCs other than erasure channels. 

3) Over erasure channels, the receiver knows exactly when it successfully recovers the in- 
formation bits and hence the right time to stop receiving coded bits. However, over other 
BIMSCs, no obvious ways (without feedback) exist to guarantee with probability 1 the 
correctness of the recovered information bits. 

In this paper, we are concerned with the construction of synchronous rateless codes for binary 
input additive white Gaussian noise (BIAWGN) channels. We propose a new ensemble of rateless 
forward error correction (FEC) codes based on serially concatenated structure, which take RS 
codes as outer codes and Kite codes as inner codes. The inner Kite codes are a special class 
of prefix rateless low-density parity-check (PRLDPC) codes, which can generate as many as 
required random-like parity-check bits. The use of RS codes as outer codes not only lowers 
down error-floors but also ensures (with high probability) the correctness of successfully decoded 
codewords. The proposed codes may find applications in the following scenarios. 

1) The proposed encoding method can definitely be utilized to construct FEC codes with any 
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given coding rate. This feature is attractive for future deep-space communications lfT4ll . 

2) The proposed codes can be considered as rate-compatible codes, which can be applied to 
hybrid automatic repeat request (H-ARQ) systems [fT31 . 

3) The proposed codes consist of a family of FEC codes with rates varying "continuously" 
in a wide range. When combined with bit-interleaved coded modulation (BICM) [16J, the 
proposed coding can be applied to adaptive coded modulation (ACM) systems ifTTl . 

4) The proposed codes can be utilized to broadcast common information over unknown noisy 
channels lfT8l . For example, assume that a sender is to transmit simultaneously a common 
message using binary-phase-shift-keying (BPSK) signalling to multiple receivers through 
different additive white Gaussian noise (AWGN) channels. Especially when these channels 
are not known to the sender, for the same reason that Fountain codes were motivated for 
erasure channels, rateless FEC codes can provide an effective solution. 

Structure: The rest of this paper is organized as follows. In Section HQ we present the 
constructions of the Kite codes as well as their encoding/decoding algorithms. The relationships 
between the Kite codes and existing codes are discussed. We emphasized that the Kite codes 
are new ensembles of LDPC codes, although any specific Kite code can be viewed as a known 
code. In Section Unl we analyze the maximum likelihood (ML) decoding performance of the Kite 
codes by applying a refined Divsalar bound to the ensemble weight enumerating function (WEF). 
In Section [TV] a greedy optimizing algorithm to design Kite codes is introduced. We also 
discuss the optimization algorithm by using density evolutions. In Section |Vj we present the 
serial concatenations of RS codes and Kite codes (for brevity, RS-Kite codes), followed by the 
performance evaluation of RS-Kite codes and construction examples in Section [VIl Section IVIII 
concludes this paper. 

Notation: A random variable is denoted by an upper case letter X, whose realization is 
denoted by the corresponding lower case letter x. We use Px(x) to represent the probability 
mass function for a discrete variable X or probability density function for a continuous variable 
X. We use Pr{^4} to represent the probability that the event A occurs. 
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II. Kite Codes 

A. Prefix Rateless LDPC Codes 

Let F q be the finite field with q elements. Let be the set of all infinite sequences over 
F q . As we know, is a linear space under the conventional sequence addition and scalar 
multiplication. A rateless linear code C[oo,fc] is defined as a fc-dimensional subspace of F^°. 
Let C[n, k] be the prefix code of length n induced by C[oo, k), that is, 

C[n, k] = |c[n] = (c , Ci, • • • , c n _i) | c[n] is a prefix of some infinite sequence c G C[oo, . 

(1) 

Clearly, C[n, k] is a linear code. If C[fc, k] (hence C[n, k],n > k) has dimension k, we call C[oo, k] 
a prefix rateless linear code. Equivalently, a prefix linear code can be used as a systematic code 
by treating the initial k bits as information bits. Furthermore, if all C[n, k] have low-density 
parity-check matrices, we call C[oo,k] a prefix rateless low-density parity-check (PRLDPC) 
code. 

B. Kite Codes 

We now present a special class of PRLDPC codes, called Kite codes. An ensemble of 
Kite codes, denoted by /C[oo,fc;p], is specified by its dimension k and a real sequence p = 
(po,Pi,--- ,pt, •••) with < p t < 1 for t > 0. For convenience, we call this sequence /?- 
sequence. The encoder of a Kite code consists of a buffer of size fc, a pseudo-random number 
generator (PRNG) and an accumulator, as shown in Fig. CD We assume that the intended receivers 
know exactly the p-sequence and the seed of the PRNG. 

Let v — (t> , v i, • ■ ■ , ffc-i) be the binary sequence to be encoded. The corresponding codeword 
is written as c = (v,w), where w = (wq, w\, • • • , w t , • • • ) is the parity-check sequence. The task 
of the encoder is to compute w t for any t > 0. The encoding algorithm is described as follows. 

Algorithm 1: (Recursive encoding algorithm for Kite codes) 

1) Initialization: Initially, set w-i = 0. Equivalently, set the initial state of the accumulator to 
be zero. 

2) Recursion: For t = 0, 1, • - • ,T — 1 with T as large as required, compute w t recursively 
according to the following procedures. 
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buffer of size k bits 




At time t, generate a random number ~ B(k; p t ) ; 
generate a parity-check sum with uniformly at random 
selected ff summands from the buffer; 



0- 



*• Wt-l — 1 



v — systematic sequence of £ bits 
w — parity-check sequence of r = n - k bits 
B(k\ p t ) — binomial distribution with k Bernoulli trials and success probability p, 

s, — parity-check sum at time t 
PRNG — pseudo-random number generator 



Fig. 1. A systematic encoding algorithm for the proposed Kite code. 



Step 2.1: At time t > 0, sampling from k independent identically distributed (i.i.d.) binary 
random variables H_ t = (H t , H t l , • ■ ■ , H t)k _i), where 

{p t , ha — 1 
(2) 
i - Pt, h t>i = o 

for < i < k; 

Step 2.2: Compute the t-th parity-check bit by w t = Wt-i + 2~2o<i<k ht,iVi mod 2. 
Remark. Algorithm 1 can also be implemented by first sampling from the binomial distribution 
B(k;p t ) to obtain an integer 9 t and then choosing 8 t summands uniformly at random from the 
buffer to obtain a parity-check sum s t , which is utilized to drive the accumulator, as shown in 
Fig. [B 

For convenience, the prefix code /C[n, k] of a Kite code /C[oo, k; p] is also called a Kite code. 
A Kite code JC[n, k] for n > k is a systematic linear code with r = n — k parity-check bits. We 
can write its parity-check matrix H as 

H = (H v , H w ) , (3) 
where H v is a matrix of size r x k that corresponds to the information bits, and H w is a square 
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matrix of size r x r that corresponds to the parity-check bits. By construction, we can see that the 
sub-matrix H v is a random-like binary matrix whose entries are governed by the p-sequence and 
the initial state of the PRNG. In contrast, the square matrix H w is a dual-diagonal matrix (blanks 
represent zeros) 



( 1 

1 1 
1 

V 



\ 



1 



(4) 



1 1/ 

Since the density of the parity-check matrix is dominated by the p-sequence, we can construct 
Kite codes as LDPC codes by choosing p t <C 1/2. If so, the receiver can perform the iterative 
sum-product decoding algorithm [4]. The iterative sum-product algorithm can be described as a 
message passing/processing algorithm on factor graphs [fl9l . A Forney-style factor graph (also 
called a normal graph [|2~0l ) representation of fC[n, k] is shown in Fig. [2] 

Now we assume that the coded bit c t at time t > is modulated in a BPSK format and 
transmitted over the AWGN channel. With this assumption, an intended receiver observes a 
noisy version y t = x t + z t , where x t = 1 — 2c t and z t is an AWGN sample at time t. The noise 
variance a 2 is assumed to be known at the receiver. We also assume that the intended receiver 
has a large enough buffer to store the received sequence for the purpose of decoding. 

Once a noisy prefix (of length n) y[n] is available, the receiver can perform the iterative sum- 
product decoding algorithm to get an estimate of the transmitted codeword c[n]. The decoding 
is said to be successful if the decoder can find an estimate c[n] within J (a preset maximum 
iteration number) iterations satisfying all the parity-check equations specified by H. If not such 
a case, the receiver may collect more noisy bits to try the decoding again. If n is the length of 
the prefix code that can be decoded successfully, we call k/n decoding rate rather than coding 
rate. The decoding rate is a random variable, which may vary from frame to frame. However, the 
expectation of the decoding rate cannot exceed the corresponding channel capacity [21 j. Given a 
signal-to-noise-ratio (SNR) of 1/a 2 , the gap between the average decoding rate and the channel 
capacity can be utilized to evaluate the performance of the proposed Kite code. A rateless code 
is called universal if it is "good" in a wide range of SNRs. It is reasonable to guess that the 
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Fig. 2. A normal graph of a Kite code. 
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universality of the proposed Kite code for given k can be improved by properly choosing the 
p- sequence. 

C. Relations Between Kite Codes and Existing Codes 

The proposed Kite codes are different from existing iteratively decodeable codes. 

1) The Kite codes are different from general LDPC codes. An ensemble of Kite codes is 
systematic, rateless and characterized by the p-sequence, while an ensemble of general 
LDPC codes is usually non-systematic and characterized by two degree distributions \{x) = 
J2i Kx 1 ^ 1 and p(x) = YliPi^ 1 ^ where K (Pi) represents the fraction of edges emanating 
from variable (check) nodes of degree i 11221 11231 . 

2) The Kite codes can be considered as serially concatenated codes with systematic low- 
density generator-matrix (LDGM) codes [[24] as outer codes and an accumulator as inner 
code. However, different from conventional serially concatenated codes, the inner code 
takes only the parity-check bits from the outer codes as input bits. An another difference 
is as follows. The generator matrices Hj (transpose of H v ) of outer LDGM codes may 
have infinite columns with random weights governed by the p- sequence rather than degree 
polynomials. 

3) The Kite codes are also similar to the generalized irregular repeat-accumulate (GeIRA) 
codes 11231 Il2~6lll2~7lll28l . However, GeIRA codes are usually specified by the repetition 
multiplicities of each information bits and the uniform interleaves 

4) As a rateless coding method, the proposed Kite codes are different from LT-codes and Raptor 
codes. For LT-codes and Raptor codes, coded bits are generated independently according to a 
time-independent degree distribution; while for Kite codes, parity-check bits are generated 
dependently. The Kite codes are more similar to the codes proposed in [29], which are 
designed for erasure channels and specified by degree polynomials. 

As an ensemble of codes, the proposed Kite codes are new. The main feature of Kite codes is 
the use of the p-sequence instead of degree distributions to define the ensemble. This has brought 
at least two advantages. Firstly, as shown in Sec. [nil the weight enumerating function (WEF) of 
the ensemble can be easily calculated, implying that the ML decoding performance of Kite 
codes can be analyzed. Secondly, as shown in Sec. [IV] Kite codes can be designed by a 
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greedy optimization algorithm which consists of several one-dimensional search rather than 
high-dimensional differential evolution algorithms fl30l ED. 

III. Maximum Likelihood Decoding Analysis of Kite Codes 

A. Weight Enumerating Function of the Ensemble of Kite Codes 

We can define the input-redundancy weight enumerating function (IRWEF) of an ensemble 
of (prefix) Kite codes with dimension k and length n as ||32l 

A(X,Z)^J2 A ^ Xizj > & 

where X, Z are two dummy variables and A\j denotes the ensemble average of the number of 
codewords c = (v, w) consisting of an input information sequence v of Hamming weight i and 
a parity check sequence w of Hamming weight j. 

Let be an input sequence consisting of £ ones followed by k — £ zeros. That is, = 

(^_^_^^_^_ 0,)- Let be the resulting parity-check sequence, which is a sample of a random 

i k-e 

sequence W}^ depending on the choice of the parity-check matrix. Given the probability 
Pr{wf° = 1} can be determined recursively as follows. 

Firstly, note that the random binary sum (see Fig. Q]for the notation s t ) = Yl Ht,i- 

o<i<e-i 

So pf^ — Yi{Sf^ = 1} can be calculated recursively as pf^ = pf^\l —p t ) + (1 —pf~^)pt for 
£ > whereby pf^ is initialized as zero. Secondly, we note that the sequence W}^ is a Markov 
process with the following time-dependent transition probabilities 

Pr{wf = | W& = 0} = 1 - pf\ Pr{M<f } = 1 | = 0} = pf\ 
Pr{wf > = | = 1} = pf\ Pr{wf = 1 | Wffl = 1} = 1 - pf\ 

We have the following two propositions. 

Proposition 1: Let A^ e \Z) be the ensemble average weight enumerating function of W} e '. 
Let a t (Z;w) be the ensemble average weight enumerating function of the prefix sequence 
W}W = {W^\wf\ ••• ,WP) ending with W t {£) = w at time t Then A®{Z) can be cal- 
culated recursively by performing a forward trellis-based algorithm (see Fig. [3] for the trellis 
representation) over the polynomial ring. 

• Initially, set 0) = 1 and a_i(Z; 1) = 0; 
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. For t > 0, 

a t (Z;l) = p ( f ) Z-a t . 1 (Z;0) + (l-p ( f ) )Z-a t ^(Z;l) 

. At time r - 1, we have A (l \Z) = a r _i(Z; 0) + a r _i(Z; 1). 

Proof: The algorithm is similar to the trellis algorithm over polynomial rings for computing 
the weight enumerators of paths [|33l . The difference is that each path here has a probability, 
which can be incorporated into the recursions by assigning to each branch the corresponding 
transition probability as a factor of the branch metric. ■ 

Proposition 2: A(X, Z) = £ ( k ] X e A®(Z). 

o<e<k \ £ I 

Proof: By the construction, we can see that the columns of H v are identically independent 
distributed. This implies that A^\Z) depends only on the Hamming weight t of v but not on 
the locations of the i ones in the sequence v. ■ 

B. Refined Divsalar Bound 

We now proceed to discuss the ML decoding performance of Kite codes. Consider the prefix 
code tC[n, k;p}. Assume that the all-zero codeword c°[n] is transmitted and y[n] is the received 
vector with y t — 1 — 2ct + z t and z t is an AWGN sample with zero mean and variance a 2 . The 
probability of ML decoding error can be expressed as 

Pr{£} = Pr |U^|> (7) 
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where E d is the event that there exists at least one codeword of weight d that is nearer to y[n] 
than c° [n] . 



Let S d = 2~2 Aij for all < d < n. Divsalar derived a simple upper bound 1134 

i,j.i+j=d 



where 



Pr{E d } < min { e - nE ^\ S d Q (v 7 ^) } , (8) 



E(5,f3 n ) = -r n {5) + ~m (0 + (1 - /3)e 2 -^) + ; _ ^ (9) 



and 7 = 5 = d/n, r n (5) = ^ and 



" - (1^1^ + (i^) 2 [(1 + 7 )> - 1] - + 7). (10) 

From © and the union bound, we have 

Pr {E} < ^min {e~ nE ^\ S d Q (v 7 ^) } • (11) 

d 

The question is, how many terms do we need to count in the above bound? If too few terms 
are counted, we will obtain a lower bound of the upper bound, which may be neither an upper 
bound nor a lower bound; if too many are counted, we need pay more effort to compute the 
distance distribution and we will obtain a loose upper bound. To get a tight upper bound, we may 
determine the terms by analyzing the facets of the Voronoi region of the codeword c°[n], which 
is a difficult task for a general code. We have proposed a technique to reduce the number of 
terms [1361 . For completeness, we include a brief description of the technique here. The basic idea 
is to limit the competitive candidate codewords by using the following suboptimal algorithm. 

Algorithm 2: (A list decoding algorithm for the purpose of performance analysis) 

51. Make hard decisions, for < t < n — 1, 

~ / °' yt>0 no, 
Vt = < • (12) 

I 1, Vt < o 

Then the channel c t — > y~t becomes a memory less binary symmetric channel (BSC) with 
cross probability 

- = / -j=e^dz. (13) 
a J J 1/(7 V2n 

52. List all codewords within the Hamming sphere with center at y of radius d* > 0. The 
resulted list is denoted as £„. 
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S3. If C y is empty, report a decoding error; otherwise, find the codeword c* G C y that is closest 
to y[n]. 
Now we define 

K = {y\c°[n} G £ £ } . (14) 

In words, the region 7?. consists of all those y[n] having at most d* non-positive components. 
The decoding error occurs in two cases under the assumption that the all-zero codeword c°[n] 
is transmitted. 

Case 1. The all-zero codeword is not in the list C y (see Fig. IU(a)), that is, y[n) G' 1Z, which 
means that at least d* + 1 errors occur over the BSC. This probability is 

Yx{y[n\in}= (flPBScil-PBScY- 1 . (15) 

t=d*+i ^ ' 

Case 2. The all-zero codeword is in the list C y , but is not the closest one (see Fig. |4] (b)), 
which is equivalent to the event {E,y[n] G 72}. Since all codewords in the list C y are at most 
2d* away from the all-zero codeword, this probability is upper-bounded by 

Pr{E,y[n]en} < Pr I (J E d , y[n] G K 1 (16) 

{d<2d* ) 

< Pr I |J j/[n] G R n I (17) 

< ^Pr{E d } (18) 

d<2d* 

< £ min {e"" B(<5,/3,7) , S d Q (v^y) } • (19) 

d<2d* 

Combining (El) and ([191) with Gallager's first bounding technique (GFBT) J37]| 

Pr{£} < Pr{E,y[n] G 72.} + Pr{y[n] £ 72}, (20) 

we get an upper bound 

Pr{E} < min {e-^^, S d Q (y/2dj) } + £ (")pW(1 - Pssc)^ (21) 
To calculate the upper bound on bit-error probability, we need to replace S d in (|2T1) by 

S'd= E (22) 

i,jH+j=d 
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Fig. 4. Graphical illustrations of the decoding error events. 

Here we have used the fact that, in the case of c°[n] ^ C y , the decoding error can contribute at 
most one to the bit-error rate (BER). 

The modified upper bound in (12TT ) has advantageous over the original Divsalar bound (fTTI) . On 
one hand, the modified upper bound in (t2TT> is less complex than the original Divsalar bound (fTTT) 
for codes having no closed-form WEFs since its computation only involves portion of the weight 
spectrum up to d < 2d*. On the other hand, if all Ai/s for i + j < D are available, we can get 
a tighter upper bound 



which can be tighter than the original Divsalar bound (fTTI ) (corresponding to d* = n). 
C. Numerical Results 

We consider a Kite code /C[2100, 1890]. For simplicity, set p t = po for all t < 210. We consider 
Po = 0.5, 0.025 and 0.015, respectively. The WEFs {S' d } are shown in Fig. |5] As we can see, the 
three curves match well in the moderate-to-high-weight region. While, in the low-weight region, 
they differs much from each other, as shown in Fig. |6] As the parameter p increases to 0.5, 
the resulting ensemble becomes a uniformly random ensemble. The refined Divsalar bounds on 
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Fig. 5. Ensemble weight enumerating functions of £[2100, 1890]. 
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Fig. 6. Detailed ensemble weight enumerating functions of £[2100, 1890]. 
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Fig. 7. Performance bounds on the ensemble Kite codes /C[2100, 1890]. 




Fig. 8. Comparisons between bounds and simulation results for p — 0.025. 



BERs are shown in Fig. [7] Shown in Fig. [8] are the comparisons between bounds and simulation 
results for p = 0.025. We have the following observations. 

1) The modified bound improves the original Divsalar bound especially in the low-SNR region. 

2) Under the assumption of the ML decoding, the performance degrades as the parameter p 
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decreases. There exists an error floor at BER around 10~ 4 for p = 0.025. At the corner of 
the floor, the performance gap (in terms of upper bounds) between the code with p = 0.025 
and the totally random code is less than 0.1 dB. 
3) The iterative sum-product decoding algorithm delivers a curve that matches well with the 
performance bounds of the ML decoding in the high-SNR region. 

IV. Design of Kite Codes 

A. Partition the p-Sequence According to Decoding Rates 

As we have observed in Section [III] that the choices of the p-sequence have effect on the 
performance of Kite codes. On one hand, to ensure with high probability that the resulting Kite 
codes are iteratively decodable LDPC codes, we must choose the p-sequence such that p t <C 1/2. 
On the other hand, to guarantee the performance, the components of the p- sequence can not be 
too small. 

The task to optimize a Kite code is to select the whole p-sequence such that all the prefix codes 
are good enough. This is a multi-objective optimization problem and could be very complex. 
For simplicity, we only consider the codes with rates not less than 0.1 and simply group the 



p- sequence according to decoding rates 


as follows 




99, 


0.9 < k/(t + k) < 1.0 




98, 


0.8 < k/(t + k) < 0.9 




97, 


0.7 < k/(t + k) < 0.8 




96, 


0.6 < k/{t + k) < 0.7 


Pt = < 


95, 


0.5 < k/{t + k) < 0.6 




94, 


0.4 < k/(t + k) < 0.5 




93, 


0.3 < k/{t + k) < 0.4 




92, 


0.2 < k/{t + k) < 0.3 




k 9i, 


0.1 < k/{t + k) < 0.2 



Then the task to design a Kite code is to select the parameters q = (g 9) q$, ■ • ■ ,q 1 ). 

B. Greedy Optimizing Algorithms 

We use a greedy algorithm to optimize the parameters q. Firstly, we choose q 9 such that the 
prefix code /C[[A;/0.9J, k] is as good as possible. Secondly, we choose q% with fixed q 9 such that 
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4 5 6 7 8 9 10 

SNR (Signal-to-Noise-Ratio) = 10log 10 (1/a 2 ) [dB] 



Fig. 9. BER performance of £[2100, 1890]. 

the prefix code /C[|_A;/0.8j , k] is as good as possible. Thirdly, we choose q 7 with fixed (qg,qs) 
such that the prefix code IC[[k/0.7\,k] is as good as possible. This process continues until q\ 
is selected. 

Let qe be the parameter to be optimized. Since the parameters qj with j > £ have been 
fixed and the parameters qj with j < £ are irrelevant to the current prefix code, the problem 
to design the current prefix code then becomes a one-dimensional optimization problem, which 
can be solved, for example, by the golden search method [|38l . What we need to do is to make 
a choice between any two candidate parameters qe and q[, which can be done (with certain BER 
performance criterion) by at least two methods. We take k = 1890 as an example to illustrate 
the main idea. 

1 ) Simulation-Based Methods: We can first simulate the BER versus SNR curves and then 
make a choice between q? and q[. For example, Fig. [^illustrates our simulations for K, [2100, 1890] . 
If the target is BER < 10~ 3 , we say that q$ = 0.025 is superior to q' 9 = 0.015. At first glance, 
such a simulation-based method could be very time-consuming. However, it is fast due to the 
following two reasons. Firstly, in this paper, our optimization target is set to be BER = 10~ 4 , 
which can be reliably estimated without large amounts of simulations. Secondly, our goal is not 
to reliably estimate the performances corresponding to the two parameters qt and q' e but to make 
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a choice between these two parameters. Making a choice between two design parameters, as an 
example of ordinal optimization problem [|39l , can be easily done by simulations. 

2) Density Evolution: We can also make a choice between q e and q' e by using density evolution. 
As shown in Fig. |2l the Kite code can be represented by a normal graph which contains three 
types of nodes: information variable nodes, parity-check variable nodes and check nodes, which 
are simply referred to as A-type nodes, 5-type nodes and C-type nodes, respectively. Since the 
connections between B-type nodes and C-type nodes are deterministic, we only need to derive 
the degree distributions for the edges between A-nodes and C-nodes. 

Node degree distributions: Assume that the parameters qj for j > £ have been optimized 
and fixed. The resulting parity-check matrix is written as = (Hv +1 \ Hw +1 ^), which is 

a random matrix of size r^ +1 ^ x (k + r^ +1 )). In our settings, for example, = [k/0.9\ — k, 
r^ s > = L^/0.8J — k, and so on. Noting that H^> is constructed from H^ e+1 ^ by adding S = 
r W _ r (i+i) ro ws, we may write it as 

Since Hw is deterministic, we only need to determine the degree distributions of nodes corre- 
sponding to Hi, . For doing so, assume that the degree distributions of nodes corresponding to 
Hv l+l ^ are, for A-type nodes, 

and for C-type nodes, 

R(t+V( x ) = Rf +1) x\ (27) 

0<i<k 

where Af +1 ^ (resp. Rf +1 ' ) ) represents the fraction of A-type (resp. C-type) nodes of degree i. In 
our settings, for example, A^ = ( r{ '^)q l g (l — q 9 ) r{9) ~' t for < i < and = ( k )ql(l — q$) k ~ l 
for < % < k. Since Aq 9 ^ > 0, there may exist nodes with no edges. However, such a probability 
can be very small for large r^. It should also be pointed out that R^ i+1 \x) only counts the 
edges of the C-type nodes connecting to the A-type nodes. For this reason, we call R^(x) the 
left degree distribution of C-nodes. 
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Let A(x) = J2 0<i<s AjX 2 and R(x) = ^2 0<i<k Ri xl De the degree distributions of nodes 
corresponding to /ft . Then we have 

A,= ^qi{l- qi ) s -\ 0<i<6 (28) 

and 

Ri= r*jqi(l-q i ) k - i , <i < k. (29) 
Then it is not difficult to verify that 

Af = J2 < < < ^ W (3°) 

and 

itf = ^ -, < fc. (31) 

Edge degree distributions: Given A^(x) and R^(x) (which can be computed recursively), 
we can find the degree distributions of edges between A-type nodes and C-type nodes as follows. 
Let X(x) = J2i an d p(x) = EjPi 3 -*" 1 be the degree distributions of edges corresponding 

to Hjf\ We have 

A, = ^V, l<i<r«> (32) 

and 

p 4 = -^-, l<z<k. (33) 

Gaussian Approximation: The density evolution [|23l is an algorithm that predicts the per- 
formance thresholds of random LDPC codes by tracing the probability density function (pdf) 
of the messages exchanging between different types of nodes under certain assumptions^. In 
the density evolution, we assume that the all-zero codeword is transmitted. It is convenient to 
assume that the input messages to the decoder are initialized by the following log-likelihood 
ratios (LLRs) 

w , fy{y\ + 1) j \ 

Mo = log -r-f-, — 7T) (34) 
Ml/ - 1) 



'Strictly speaking, the density evolution can not be applied here to optimize the parameters because we are constructing 
codes with a fixed dimension k, the parameter qi is evidently dependent of the code dimension, and the code graph is only 
semi-random. 
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where fy is the conditional pdf of Y given that +1 is transmitted. It has been proven [23] that, for 
the BPSK input and continuous output AWGN channel, M is a Gaussian random variable having 
mean fj, = 2 /a 2 and satisfying the symmetry condition a 2 = 2p . It was further shown [40] 
that all the intermediate messages produced during the iterative sum-product algorithm can be 
approximated by Gaussian variables or mixture of Gaussian variables satisfying the symmetry 
condition. This means that we need to trace only the means of the messages. 

For Kite codes shown in Fig. [2l the density evolution using Gaussian approximations are 
slightly different from that general LDPC codes. To describe the algorithm more clearly, we 
introduce the following notation. 

/i the mean of initial messages from the channel; 
/i^^ c ^ the mean of messages from A-type nodes of degree % to C-type nodes; 
yS B ^ c > the mean of messages from B-type nodes to C-type nodes; 
fi\ C ^ A ^ the mean of messages from C-type nodes of left degree % to A-type nodes; 
j[( c ^ A ) the average mean of messages from A-type nodes to C-type nodes; 
fi\ ' the mean of messages from C-type nodes of left degree i to £>-type nodes; 
^j(c^b) ^ avera g e mean f messages from C-type nodes to 5-type nodes. 

Slightly different from [1401 but following BP . we define 



We may use the following algorithm to predict the performance threshold for the considered 
parameter under the iterative sum-product decoding algorithm. 

Algorithm 3: (Density evolution using Gaussian approximations for Kite codes) 

1) Input: The degree distributions of nodes A(x), R(x); the degree distributions of edges \(x), 
p(x); target T b of BER and absolute difference A b of BERs for two successive iterations; 

2) Initializations: Set e = J ^ -^=e~( y -^ 2 /^dy, Jt^^ = and Ji^ B ^ = 0; 

3) iterations - Repeat: 

Step 3.1: From A-type nodes to C-type nodes, 




(35) 



Ho + (t-l)-p {C ^ A \l<i<r; 



(36) 



Step 3.2: From B-type, nodes to C-type nodes, 



Ho + A* 



(37) 
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Step 3.3: 

a) From C-type nodes to A-type nodes, 

= ^V^ C) ) (^\H^ C) )J j ,l<j<k; (38) 

n {c -* A) =izp^r A) , (39) 

b) From C-type nodes to 5-type nodes, 

= r l \^ B - C) ) [ £ C) ) ) )iO<j<k; (40) 



-(c^^Rjtf^; (41) 

3=0 

Step 3.4: Make decisions, for < i < r, define = /i + ipZ^^^; compute 

_^ =e -(y-^) 2 /(^) dy (42) 

and 

r 

£ ' = £a^; (43) 

if e' < T b or \e — e'\ < A b , exit the iteration; else set e — e' and go to Step 3.1. 
Remark. Note that we have ignored the effect caused by the margin of the subgraph consisting 
of £>-type nodes and C-type nodes. Also note that, different from the density evolution for general 
LDPC codes, degree distributions of nodes are also involved here. 
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Fig. 10. Performance of the constructed Kite code with k = 1890. From left to right, the curves correspond to rates 0.1, 0.2, 
0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9, respectively. 



C. Construction Examples 

In this subsection, we present a construction example. We take the data length k = 1890. A 
Kite code /C[oo, 1890; p] can be constructed using the following p-sequence 



99 


= 0.0249, 


0.9 < k/(t + k) 


< 1.0 


98 


= 0.0072, 


0.8 < k/(t + k) 


< 0.9 


97 


= 0.0045, 


0.7 < k/(t + k) 


< 0.8 


Qe 


= 0.0034, 


0.6 < k/(t + k) 


< 0.7 


95 


= 0.0021, 


0.5 < k/{t + k) 


< 0.6 


94 


= 0.0016, 


0.4 < k/it + k) 


< 0.5 


93 


= 0.0010, 


0.3 < k/(t + k) 


< 0.4 


q-2 


= 0.0006, 


0.2 < k/(t + k) 


< 0.3 


9i 


= 0.0004, 


0.1 < k/it + k) 


< 0.2 



The simulation results are shown in Fig. [TO] For comparison, the SNR for the simulation at 
BER = 10~ 4 and the SNR thresholds estimated by the Gaussian-approximation-based density 
evolution (GA DE) are listed in the Table H As we can see, the difference between the density 
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TABLE I 

The SNR for the density evolution and simulation at BER = 10 



Code rate 


GA DE (dB) 


Simulation (dB) 


0.9 


6.49 


7.0 


0.8 


4.84 


5.6 


0.7 


3.65 


4.4 


0.6 


2.57 


3.2 


0.5 


1.48 


2.1 


0.4 


0.31 


0.9 


0.3 


-1.04 


-0.5 


0.2 


-2.84 


-2.4 


0.1 


-5.67 


-5.3 



evolution and the simulation is less than 1 dB. 

V. Serial Concatenation of Reed-Solomon Codes and Kite Codes 
A. Encoding 

Naturally, the proposed encoding method can be utilized to construct fixed-rate codes. How- 
ever, we found by simulation that, in the range of moderate-to-high rates, the constructed fixed- 
rate codes in this way suffer from error-floors at BER around 1CT 4 , as shown in Fig. [T0l The 
error floor is caused by the possibly existing the all-zero columns in the randomly generated 
parity-check matrices. To lower-down the error-floor, we may insert some fixed-patterns into the 
matrices, as proposed in 02). We also note that, in the rateless coding scenario, the receiver 
must find a way to ensure the correctness of the successfully decoded codeword. This is trivial 
for erasure channels but becomes complex for noisy channels. For erasure channels, a decoded 
codeword is correct if and only if all information bits are recovered uniquely. For AWGN 
channels, no simple way to ensure with probability one the correctness of the decoded codeword 
at the receiver. In order to lower down the error- floors and ensure with high probability the 
correctness of the decoded codewords, we employ the serially concatenated coding system 
proposed by Forney [1J. As shown in Fig. [CD the outer code is a systematic RS code of dimension 
K and length N over F 2 m, while the inner code is a Kite code. Note that no interleaver is required 
between the outer encoder and the inner encoder. For convenience, such a concatenated code is 
called an RS-Kite code. 
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u 


Systematic 




Systematic 


c 


► 


RS encoder 


► 


Kite encoder 


► 



Fig. 11. Serial concatenation of RS codes and Kite Codes. 

Let u be the binary data sequence of length ImK (£ > 0). The encoding procedure is described 
as follows. First, the binary sequence u is interpreted as a 2 m -ary sequence of length £K and 
encoded by the outer encoder, producing £ RS codewords. Then these £ RS codewords are 
interpreted as a binary sequence v of length £mN and encoded using a Kite code, resulting in 
a potentially infinite coded sequence c. 

B. Decoding 

We will use Berlekamp-Massey (BM) algorithm [43] for the outer decoder and iterative sum- 
product algorithm for the inner decoder. As we know, if the number of errors in a noisy RS 
codeword is not greater than t max — \_(N — K)/2\, the BM algorithm will find the correct 
codeword. On the other hand, if the number of errors in a noisy RS codeword is greater than 
t max , the BM algorithm may claim a decoding failure or make a miscorrection. We call the BM 
decoding successful whenever the BM algorithm finds a codeword within the Hamming sphere 
of radius t max around the received "codeword" from the inner decoder. Since the miscorrection 
probability can be made very small by properly setting the parameters of the RS code [44J[45J, we 
will treat the successfully decoded RS codewords as "side information" (prior known information) 
for the inner decoder. For £ > 1, such known information can be fed back to the inner decoder to 
implement iterations between the outer decoder and the inner decoder [46] [47). Let Pi ij) {u ^ u} 
be the block error probability after the j-th iteration. Obviously, we have Pr^ +1 ^{u ^ u} < 
since such iterations never change a successfully decoded codeword from a correct 
one to an erroneous one. 

The decoding algorithm for the RS-Kite codes with incremental redundancy is described in 
the following as well as shown in Fig. [T2l 

Algorithm 4: (Decoding algorithm for RS-Kite codes) 

1) Initialization: Properly choose two positive integers r and A r . Set n = k + r . Set ji = 
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Initialization: 
Choose r > 0, A r > 0; 
Set n = k + r ; 
, Set yi = 0, i = 0,1, — ,1-1 



Receiving more noisy coded bits until 
the prefix y[n] is available 



z 



Decoding the prefix Kite code /C[n,k] 
using iterative sum-product algorithm 



z 



Seti = 0;5= 1; 




No 



Decoding the i th RS code using 
BM algorithm 



Feedback 
the i th (yi = 1) 
decoded RS 
codeword as 
known information 



No 



Is the decoding\^No^ 

successful? 



Yes 



Yi= 1; 5 = 



z 



V H 



Increase i by 1 




No 




No 



Increase n by A r 




Yes 



c 



stop 



3 



Fig. 12. A flowchart of the decoding algorithm for the proposed serially concatenated codes. 
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for < i < £. 

2) Iteration: While n < k + T, do the following procedures iteratively. 

Step 2.1: Once the noisy prefix y[n] is available in the receiver, perform the iterative sum- 
product decoding algorithm for the prefix Kite code K,[n, k] until the inner decoder is 
successful or the iteration number exceeds a preset maximum iteration number J. In this 
step, all successfully decoded RS codewords (indicated by 7, = 1) in previous iterations 
are treated as known information; 

Step 2.2: Set 5 = 1. For < % < i, perform the BM decoding algorithm for the i-th noisy 
RS codeword that has not been decoded successfully (indicated by 7$ = 0) in previous 
iterations. If the BM decoding is successful, set 7, = 1 and 5 = 0; 

Step 2.3: If all RS codewords are successfully decoded (i.e., 7* = 1 for all < % < i), stop 

decoding; else if 5 — 1, increase n by A r ; else keep n unchanged. 
Remark. In the above decoding algorithm for RS-Kite codes, the boolean variable 5 is 
introduced to indicate whether or not increasing the redundancy is necessary to recover the 
data sequence reliably. If there are no new successfully decoded RS codewords in the current 
iteration, increase the redundancy and try the inner decoding again. If there are new successfully 
decoded codewords in the current iteration, keep the redundancy unchanged and try the inner 
decoding again by treating all the successfully decoded codewords as known prior information. 

VI. Performance Evaluation of RS-Kite Codes and Construction Examples 

In this section, we use a semi-analytic method to evaluate the performance of the proposed 
RS-Kite code. Firstly, we assume that the performance of the Kite codes can be reliably estimated 
by Monte Carlo simulation around BER = 1CT 4 ; secondly, we use some known performance 
bounds for RS codes to determine the asymptotic performance of the whole system. 

A. Performance Evaluation of RS Codes 

The function of the outer RS code is to remove the residual errors in the systematic bits of 
the inner code after the inner decoding. Hence, we only consider the systematic bits of the inner 
code. Let P b be the probability of a given systematic bit at time t being erroneous after the inner 
decoding, that is, P b = Pr{v t 7^ v t }. By the symmetric construction of Kite codes, we can see 
that P b is independent of the time index t (0 < t < L v — 1). We assume that P b can be estimated 
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reliably either by performance bounds or by Monte Carlo simulations for low SNRs. To apply 
some known bounds to the outer code, we make the following assumptions on the error patterns 
after the inner decoding. 

• The outer decoder sees a memoryless "channel". This assumption is reasonable for large i. 
For small £, the errors in the whole inner codeword must be dependent. The dependency 
becomes weaker when only systematic bits are considered. 

• The outer decoder sees a g-ary symmetric "channel". This assumption can be realized by 
taking a random generalized RS code f|4Tfl as the outer code instead. That is, every coded 
symbol from the outer encoder is multiplied by a random factor «j G ¥ q — {0} prior to 
entering the inner encoder. Accordingly, every (possibly noisy) decoded symbol from the 
inner decoder is multiplied by the corresponding factor a~[ prior to entering the outer 
decoder. 

The "channel" for the outer encder/decoder is now modelled as a memoryless g-ary symmetric 
channel (QSC) characterized by Y = X + E, where X G ¥ q and Y G ¥ q are transmitted and 
received symbols, respectively. Their difference E G ¥ q is a random variable with probability 
mass function as 

P c = Pr{E = 0} = (1 - P b ) m (45) 

and 

P e 4 Pr{E = a}= 1 " (1 ~f b)m (46) 

q- 1 

for qgF ? - {0}. The decoding error probability P err of the BM algorithm is given by 

(47) 

For the coding system with t > 1 RS codewords, successfully decoded RS codewords will 
be fed back as prior known information to the inner decoder. Hence, we need to evaluate the 
miscorrection probability of the outer code. Let {Ad, < d < N} be the weight distribution of 
the outer code. It is welLknown that KQ 

A d = ( " ] (q-1) ( T ] q d ~ dm ™~\ (48) 
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Fig. 13. Probability of decoding error for the RS code Cio24[1023, 1000] using bounded distance decoding. 



where d min = N — K + 1. The miscorrection probability can be calculated as [45 j 

J2 A 4 d ) \ d ) \ N d ) (Q-iyiQ-^P^P?-^, (49) 



mis 



i J \ J J \ h 

where the summation is for all d,i,j,h > such that d min < d < N, i < d, j < d — i, 
h < N — d, i + j + h > t max and d — i + h < t max - Under the worst case assumption that 
Pi, = 0.5, the miscorrection probability is upper bounded by , iPBl . 

B. Construction Examples 

We take RS code Ci 24[1023, 1000] over F 102 4 as the outer code. This code can correct up to 11 
symbol-errors. The performance of this code is shown in Fig. [13] and Fig. Q31 respectively. Also 
shown in Fig. [I4]is the McEliece-Swanson bound [|44l . This computations can be used to predict 
the performance of the RS-Kite codes. For example, if the inner Kite code has an error-floor at 
BER = 10" 4 , then the outer RS code Ci 02 4[1023, 1000] can lower down this error-floor to 10~ 10 
with miscorrection probability not greater than 10~ 20 . This prediction is very reliable for large 
I 

We set t = 5. Then the length of the input sequence to the inner code is 51 150. We take Kite 
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McEliece-Swanson bound 

— e— Eq. (49) 




1 2 3 4 5 



- |0 9,</ P b> 

Fig. 14. Probability of miscorrection for the RS code Cio24[1023, 1000] using bounded distance decoding. 




Fig. 15. The average decoding rates of the Kite codes. The data length k — 50000. 
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code /C[oo, 51150; p] as the inner code. The p-sequence is specified by 





= 0.00084, 


0.9 < 


k/(t + k) 


< 


1.0 


Qs 


= 0.00020, 


0.8 < 


k/(t + k) 


< 


0.9 


97 


= 0.00015, 


0.7 < 


k/(t + k) 


< 


0.8 


<16 


= 0.00009, 


0.6 < 


k/(t + k) 


< 


0.7 


q$ 


= 0.00006, 


0.5 < 


k/(t + k) 


< 


0.6 


?4 


= 0.00006, 


0.4 < 


k/(t + k) 


< 


0.5 


?3 


= 0.00004, 


0.3 < 


k/(t + k) 


< 


0.4 




= 0.00002, 


0.2 < 


k/(t + k) 


< 


0.3 


9i 


= 0.00001, 


0.1 < 


k/(t + k) 


< 


0.2 



The simulation results are shown in Fig. [15] where the error probability for each simulated 
point is upper-bounded by P mis because the receivers will try decoding with increasing redundan- 
cies until all RS codewords are decoded successfully. In our simulations, we have not observed 
any decoding errors. It can be seen from Fig. [15] that the gaps between the average decoding 
rates and the capacities are around 0.1 bits/BPSK in the SNR range of —3.0 ~ 9.0 dB. To our 
best knowledge, no simulation results were reported in the literature to illustrate that one coding 
method can produce good codes in such a wide range. We have also observed that there is a 
"singular" point at rate of 0.9. This is because we have taken 0.9 as the first target rate. If we add 
one more target rate of 0.95, we can make this curve more smooth. But the other g-parameters 
need to be re- selected and the whole curve will change. 

VII. Conclusion 

In this paper, we have proposed a new class of rateless forward error correction codes which 
can be applied to AWGN channels. The codes consist of RS codes and Kite codes linked in a 
serial concatenation manner. The inner Kite codes can generate potentially infinite parity-check 
bits with linear complexity. The use of RS codes as outer codes not only lowers down the 
error-floors but also ensures with high probability the correctness of the successfully decoded 
codewords. A semi-analytic method has been proposed to predict the error-floors of the proposed 
codes, which has been verified by numerical results. Numerical results also show that the 
proposed codes perform well over AWGN channels in a wide range of SNRs in terms of the 
gap to the channel capacity. 
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